ci(release): verify selected prepare artifacts - #25
Merged
farhan-syah merged 2 commits intoJul 28, 2026
Merged
Conversation
Split release verification into two focused checks: verify_prepare_run.sh confirms the selected Release Prepare run is a successful, tag-pushed build of the exact release commit before its artifacts are ever downloaded, and verify_release_artifacts.sh now only checks the downloaded attachment set against the expected archive list. Add regression tests for both verifiers and run them with shellcheck in CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reject a manually selected Release Prepare run unless it is a successful run of
.github/workflows/release-prepare.ymlfor the exact commit referenced by therequested release tag, and unless the downloaded artifacts are exactly the five
versioned
pagedb-fsckarchives the release publishes.This is the deliberately narrow follow-up to the useful portion of #24. It does
not add a provenance document, a shared prepare gate, locked publication,
transactional version stamping, release documentation, action pinning, or
literal workflow-structure tests.
Problem
The
Releaseworkflow accepts a maintainer-suppliedprepare_run_idand usesactions/download-artifactto retrieve everyfsck-*artifact from that run.The action already scopes the run lookup to the current repository, so an
additional repository field would not be an independent check. The version in
each expected archive name already covers the requested tag version.
Two independent facts remain worth checking:
resolves to.
nothing else matching the release artifact pattern.
Without those checks, a valid run ID can point to a different successful
workflow or tag build.
fail_on_unmatched_filesonly proves that at least onedownloaded file matches
artifacts/*; it does not prove that all five targetarchives are present or that they came from the tag commit being released.
Change
The two facts are checked by two scripts, ordered so the provenance decision is
made before any archive reaches the runner.
scripts/ci/verify_prepare_run.sh <run.json> <commit>runs first. Thegithub-releasejob reads the selected run through GitHub'srepository-scoped
GET /repos/{owner}/{repo}/actions/runs/{run_id}endpoint viagh api, then requires the returned run to have:head_shaequal to the commit checked out from the requested tag;.github/workflows/release-prepare.yml;push;completed;success.Fields are compared one at a time rather than as a single boolean, so a
rejection names the field that disagreed — the maintainer needs to know whether
they picked the wrong run, the wrong tag, or a run that has not finished yet.
Because this step precedes
actions/download-artifact, a wrong, unfinished, orforeign run is rejected without its archives ever being fetched or unpacked.
scripts/ci/verify_release_artifacts.sh <artifact-dir> <version>runs afterthe download and requires exactly these five regular, non-symlink files for the
validated version:
Both directions are checked by name: every expected archive must be present, and
every present entry must be expected. A cardinality comparison would be shorter
but would let one absence and one intruder cancel out, and it reports "wrong
number of files" where the useful message is which file.
A missing target, an extra downloaded attachment, a different version, a
different workflow, an unsuccessful prepare run, or a different tag commit
fails before
softprops/action-gh-releaseruns.The existing job-level
actions: readpermission is sufficient; no newrepository permission or secret is introduced.
ghandjqare both preinstalledon the runner image.
Scope of the guarantee
These checks establish provenance and completeness, not content integrity.
Nothing re-derives a digest for the archives. The archives are trusted on the
strength of a run pinned to the tag commit, the prepare workflow file, and a
successful conclusion. That boundary is stated at the top of
verify_prepare_run.shso a future reader does not mistake it for an integritycheck.
Independence and retry behavior
The crates.io stage is unchanged. It still depends only on the existing
validate-versionjob, builds from the tag checkout, and can be retried afterthe prepare artifacts expire.
Only
github-releaseconsumes the prepared binaries, so only that stageperforms this artifact check. The existing manual stage controls remain intact:
a crates.io outage does not block GitHub Release creation, and a GitHub Release
failure does not require republishing the crate.
The prepare workflow is also unchanged. It continues to build and retain the
same five artifacts for 14 days; no extra upload job or provenance artifact is
added.
Tests
scripts/ci/test_release_verifiers.shis a compact generated-data regressiontest rather than tracked fixture directories or workflow-text greps. It performs
no network access and asserts observable verifier behavior only — never YAML
layout, job names,
needssyntax, or literal workflow strings.Every negative case asserts the reason for the rejection, not merely a nonzero
exit, so a verifier that breaks in an unrelated way — bad arity, a typo in a
guard, a missing dependency — cannot pass by failing for the wrong reason.
Twenty cases cover:
head_sha; an unsuccessfulconclusion; a run still in progress; a different workflow path; a
workflow_dispatchevent; an absent field; malformed JSON; an absent metadatafile; a malformed commit argument;
attachment; a missing archive at full attachment count; an absent archive; a
symlinked archive; a directory in place of an archive; a version mismatch; a
malformed version argument; an absent artifact directory.
The missing-archive case renames rather than deletes, keeping the attachment
count at five so the by-name check is genuinely exercised.
The suite was mutation-tested to confirm it detects the failures it claims to.
Removing the missing-archive check produces three failures; removing the
head_shaprovenance pin is caught by the differing-commit case.Both scripts are held to
shellcheckby a new lint step, so they stay cleanrather than relying on a one-time local run.
Local verification:
The verifiers were also exercised against the repository's real successful
Release Preparerun29916913669for
v0.1.0-beta.3. GitHub reported head6e01fcbad579af61657c9f5367b6cbf425e5b976, and the run still retains exactlythe five expected platform archives. Passing the live API response to
verify_prepare_run.shproducedprepare run: ok, and it correctly rejected thesame response against a mismatched tag commit. Downloading those archives and
reproducing the job's
merge-multiple: trueflat layout producedrelease artifacts: ok, and adding a stray attachment to that directory wasrejected. This read-only proof did not dispatch a workflow, create a release, or
publish a crate.
Zizmor's pedantic, low-or-higher comparison reports the same 48 pre-existing
findings on current
mainand this branch: 33 unpinned action uses, 10artipacked, four template-injection findings, and one undocumented-permissionfinding. This PR introduces no additional finding. The unpinned action uses are
intentionally reserved for a separate SHA-pinning PR with Dependabot ownership.
Scope
Base:
031db94b4184f464d2dac1c901da05688fd2f7deHead:
452fc883031de8aa1fa27e32bf36225b294b9dc0Five files change, with no Rust source, package manifest, lockfile, public API,
on-disk format, benchmark, or release-documentation change. Runtime benchmark
comparisons are not applicable because the change executes only in the manual
GitHub Release workflow and its shell regression test.